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Concept mapping: evaluating the Language Arts Methods course 

Sylvia Read 1 

Abstract: In this inquiry, I examine the evidence of student learning in an 
elementary education language arts methods course. Students completed concepts 
maps that represented their understanding of effective writing instruction at the 
beginning end of the course. Eighty-one pairs of concepts maps were scored 
according to established methods. Students included 122% more concepts on 
their post-course concepts maps; links between concepts increased by 134%. I 
also analyze a typical student ’s beginning- and end-of-course maps, which shows 
that the student’s knowledge base increased both in terms of number and 
specificity of concepts. 

Keywords: teacher education, language arts methods, writing pedagogy, 
elementary methods courses. 

Recent criticism of teacher education (Kirby, McCombs, Barney, and Naftel, 2006) has 
prompted teacher educators to take a closer look at the effectiveness of their programs (Jones and 
Vesilind, 1996), and researchers have begun to focus on how to measure preservice and inservice 
teacher knowledge (Moats and Foorman, 2003; Reutzel, Dole, Sudweeks, Fawson, Read, Smith, 
Donaldson, Jones, Herman, 2007; Phelps and Schilling, 2004). 

In the area of knowledge of language and literacy concepts and methods, we know that 
teachers have gaps in their understanding (Moats and Foorman, 2003). Specifically, in the area 
of writing instruction pedagogy, teachers themselves acknowledge that they are uncomfortable 
with teaching writing largely because they lack confidence as writers themselves, which they 
often attribute to the writing instruction (or lack of) they received during their own K-12 
education (Graves, 2002; Murphy, 2003; Napoli, 2001). 

Since my goal as a teacher educator is to send teachers out into the public school with the 
pedagogical content knowledge to teach language arts, I need a way to evaluate the effectiveness 
of my language arts methods course for pre-service teachers. I need to know, at the end of my 
course, what, and how well, my students have learned. To determine this, I need to match 
assessment to course goals. 

For about three years, I’ve been adjusting my course content, but I always felt the pull to 
add goals. After reading about the “inverted pyramid” decision progress (Bass, 1999), I have 
been able to narrow the focus of the course. Briefly, Bass describes this inverted pyramid 
decision process as a series of questions he asked himself: 

• What were the four or five learning goals that I had for students in a particular 
course? 

• What did I really believe (and what did I know) about what percentage of students 
were achieving all of the goals, some of the goals, one or two of them? 

• If I had to pick one of these learning goals or outcomes as the one thing that 
students would retain from this course after leaving it, what would it be? 
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• Thinking about that one goal, then could I honestly say that I spent the most 
amount of time in the course teaching to the goal I valued most? (pp. 4-5) 

For my course, I decided that I would spent the most amount of time teaching about and 
demonstrating effective methods of spelling and writing instruction. The next logical step was to 
determine a method for measuring whether or not my students reached those goals. Traditional 
forms of assessment do not capture well the growth in their knowledge of pedagogical content 
that I hoped to measure. 

Some teacher educators use multiple choice tests because they are considered 
“objective.” Some use performance assessments, such as asking students to apply their learning 
by producing, teaching, and reflecting upon lessons taught during practicum experiences. 
Because instructional methods courses in teacher education aim to teach content pedagogy, 
methods teachers need ways to measure students’ pedagogical content knowledge. Multiple 
choice tests are a way to measure content knowledge, whereas assessing students’ lesson plans 
are a way to evaluative their pedagogical content knowledge. Through a short answer, multiple 
choice test I can efficiently measure some content knowledge, but knowledge of content 
pedagogy (Shulman, 1987) is embedded in cognitive structures that are difficult to see or 
describe. Moreover, some students may come with more or less well-developed ideas about 
effective writing instruction based on their own experiences with writing instruction (good, bad, 
and indifferent) that have occurred throughout their educational careers. These experiences 
powerfully influence their ideas about writing and teaching writing. I needed a way to assess 
their understanding before the class began and at the end of the course. Concept maps provide 
another way to measure both content knowledge and pedagogical content knowledge in one 
integrated artifact. For a multiple choice examination, students may perform well if they have 
memorized facts, but in order to construct a concept map, they need to have an integrated sense 
of how concepts relate to each other, which requires deeper understanding than mere 
memorization. 

I. Review of the Literature. 

In terms of writing instruction, the kind of information that students need is both content 
knowledge and pedagogical content knowledge, which is largely procedural in nature (Shulman, 
1987). Concept mapping allows a student to represent both their semantic long-term memory 
(Jacobs-Lawson and Hershey, 2002) and their procedural knowledge (Sims-Knight, Upchurch, 
Pendergrass, Meressi, Fortier, Tchimev, VonderHeide, and Page, 2004). 

Concept maps are graphic representations of concepts linked and arranged according to 
the students’ understanding of the relationship of the concepts and thus are a window into 
students' cognitive structures or schema (Novak, 1998). Based on research on the structure of 
knowledge, terms like “network” and “web” are metaphors we use to talk about how concepts 
are connected to other concepts in simple linear chains or in vertical hierarchical relationships. 
Concept maps are visual representations of those networks of ideas. Concepts are written in 
nodes (bubbles) and relationships between them are shown with lines that can be explained 
further using linking words written on the lines. Concept maps can be useful for identifying 
students' misconceptions, for developing their understanding, and for assessing their 
understanding summatively (Bolte, 1999). They are also useful as tools for deep learning 
because they help students connect new ideas to prior knowledge (Williams, 2004). Concepts 
maps are a valid and reliable measure of what students understand (Shavelson and Ruiz-Primo, 
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2000) and have been used in many subject areas, including biology (Mintzes, Wandersee, and 
Novak, 2001), science education (Markham and Mintzes, and Jones, 1994), math (Bolte, 1999), 
political science (Parkes, Zimmaro, Zappe, and Suen, 2000), nursing education (Williams, 2004), 
psychology (Jacobs-Lawson and Hershey, 2002), statistics (Lavigne, 2005), and medical 
education (West, Park, Pomeroy, and Sandoval, 2002). Most significantly, for my purposes, 
concept mapping has been used in teacher education (Artiles and McClafferty, 1998; Beyerbach, 
1988), but not to assess preservice students’ understanding of subject area teaching methods. 

II. Theoretical Framework. 

Kegan’s constructive-developmental theory of adult development (2000) is the larger 
theory that informs this research. In Kegan’s terms, we want preservice teachers to be making 
progressive development toward more complex constructions of their knowledge about effective 
teaching. In order to be able to do that, we must know what preconceived notions our students 
bring with them into the classroom. What are their current constructions of knowledge about 
teaching, about effective teaching, about the effective teaching of writing? Then, the form of that 
knowledge must undergo a change in form, a transformation, if we are to say that 
transformational learning has occurred. If students only add to their set of knowledge, then the 
learning is informational, which is worthwhile, but it is not transformational. The learning that 
we expect preservice teachers to experience in methods courses is a meaning-constructing task; 
they cannot merely be the recipients of knowledge by transmission. 

The theoretical framework for this research assumes the validity of Shulman’s notion of 
pedagogical content knowledge (1987). Content knowledge is subject matter knowledge that 
teachers possess. Pedagogical content knowledge is the unique blend of content knowledge and 
pedagogy that results when teachers organize, represent, and adapt topics, problems and issues 
for diverse students and when they present those topics, problems, and issues during instruction. 

The current study is an example of practitioner research (Borko, Liston, Whitcomb, 
2007), which attempts to examine practice from the inside by the teacher educator, who 
embodies a dual role of both teacher and researcher. Practitioner research is intentional because 
it is planned and deliberate, rather than spontaneous. It is also systematic because the researcher 
gathers information in an organized manner, keeps records, and analyzes the data collected in 
multiple ways. Practitioner research falls within the tradition of scholarship of teaching, an area 
of research that opens the classroom door so that teaching becomes “community property” 
(Shulman, 2004). Practitioner research should be available for public critique and review, which 
in turn makes it possible for other practioners to make use of it in their contexts. 

II. Purpose and Methods. 

Though my main intention was to evaluate the effectiveness of the course I teach and the 
way that I teach it, I hope that the method that I have used to evaluate my teaching will prove 
useful to others who want to know if their students are truly learning — if their teaching is truly 
effective. Although the Praxis and other tests of teacher knowledge can be used to measure the 
outcomes of teacher education, the results of these tests are not specific enough to be used to 
evaluate students’ learning in any particular course. By using concept mapping as a pre- and 
post-measure of student learning, I sought to determine the degree of growth in the depth and 
complexity of students’ knowledge in terms of the specific curriculum of the course. 
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I taught the students how to make a concept map by explaining, showing examples, and 
modeling. I then directed the students to create a map of their initial understanding of what 
constitutes effective writing instruction. Use of concept maps to assess students' understanding 
should be embedded in the instructional process, not just added on at the end (Mintzes, 
Wandersee, Novak, 2001). Accordingly, throughout the course, I used concept maps as part of 
instruction to explore subtopics of effective writing instruction (e.g., effective spelling 
instruction). 

Since the purpose of this study was to evaluate the effectiveness of this course, the 
students created a concept map at end of the course. Although I scored the maps, the scores were 
not part of their course grade. I compared the scores on this final concept map with the students' 
scores for their initial maps in order to measure the growth of their understanding over the course 
of the semester. 

Scoring procedures vary, but I used a simplified form of the method recommended by 
Novak and Gowin (1984) in which nodes and lines are counted. Walker and King's study (2003) 
of concept mapping as a form of student assessment also employed this scoring method. Two 
scorers scored a subset of the maps and had an interrater reliability of .98. See Figure 1, which 
shows the scoring guide. 

Nodes (Concepts) 1 pt. each 

• Don’t count redundancies 

• Don’t count irrelevant issues 

Labeled Lines/Li nk s 

Validity Link Score 

• invalid or misconceived link label = 0 pts. 

• partially valid, general or imprecise link = 1 pt. 

• valid, precise, clearly stated link = 2 pts. 

Line/Node ratio Validity ratio 

Figure 1: Concept map scoring guide. 

III. Results and Discussion. 

Eighty-one pre-course concept maps and 81 post-course concept maps were scored. A 
paired-samples t test was conducted to evaluate whether the post-course concept map scores 
were statistically significantly higher than the pre-course concept map scores. The number of 
concepts that students included in their concept maps increased significantly (t(80) = 15.04,/) < 
0.01). The number of labeled connections that students made among concepts increased 
significantly (t(80) = 8.42, p < 0.01). The validity of the connections was assessed as well. 
Connections that were not labeled received one point, but connections that were labeled received 
two points. This validity of connections score also increased significantly (t(80) = 12.34, p < 
0 . 01 ). 

From these gain scores, I could conclude that students' knowledge of effective writing 
instruction increased, but this would be misleading and insufficient. A closer examination of the 
nature of students' pre-course concept maps shows that their initial understandings of effective 
writing instruction were vague. Some of the concepts they listed in their pre-course concept 
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maps were related to general ideas about effective instruction and learning, but not specific to 
effective writing instruction (e.g. instruction should be varied, allow for growth, build upon past 
instruction). However, their post-course concept maps included specific concepts that they 
encountered during the course. 

Specific concepts addressed in the course were: 

• word study that incorporates word sorting and differentiated instruction based upon 
spelling assessments 

• genre-based writing instruction 

• elements of writing workshop: daily lessons, independent writing time, and sharing 

• writing process 

• gradual release of responsibility model (called IMSCI, explained below) 

• writing assessment and instruction based upon analytical scoring (6 traits) 

As I examined the students’ post-course maps, I looked for evidence that these major concepts 
were included. 

Let’s look at an example of a pre-course concept map (see figure 2). I chose this 
particular map because it typified a middle range of performance on the concept mapping task. 
This student’s map indicates that she conceives of effective writing instruction as something that 
requires thought and interest, should emphasize creativity, and address conventions of grammar, 
spelling and proper format. These are concepts that she probably has acquired largely through 
personal experience. Her map approximates an understanding of effective writing instruction that 
matches her development as a preservice teacher. No part of her map is “incorrect,” but other 
than her inclusion of specifics about conventions, the rest is nonspecific. Typically, students who 
possess general knowledge about a topic produce concepts maps that are wide, but not deep. One 
would expect that after students have developed a more complex, sophisticated knowledge base, 
their maps would reflect both the breadth and depth of that knowledge. 
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Figure 2: Student’s pre-course concept map. 

When we examine her end-of-course concept map (see figure 3), the level of specificity 
is much greater, reflecting an increase in the breadth and depth of her knowledge base. Her map 
now indicates that she understands effective writing instruction as an enterprise that should 
match instruction to assessment. She connects this to how feedback can be correlated with a 
specific standard, such as her new knowledge of the stages of spelling development. Her map 
also indicates that she understands how to differentiate instruction through the application of a 
scaffolding model (IMSCI). Each element of the scaffolding model has examples with detail, 
enumerating specific instructional practices. For example, she explains that modeling should 
focus students’ attention on the goal of the activity and that modeling should help eliminate 
students’ fear of failure because they’ve seen it done at least once. Her map doesn’t provide 
detail about how to teach the traits of writing, but she does enumerate them correctly. Finally, 
her map indicates an understanding of the affective dimensions of writing instruction, which we 
had discussed in class. Specifically her map indicates that effective writing instruction should 
provide students with the tools to write on their own successfully, the confidence that they can 
write, and opportunities to leam from their mistakes without feeling dumb. 
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Figure 3: Student’s end-of-course concept map. 

What happened in this course to bring about changes in students’ thinking about effective writing 
instruction? What learning experiences did the students have? In order to discuss what students 
experienced in this course, it’s necessary to explain the two main foci of my instruction. 

The first focus is the theory of the gradual release of responsibility, which I explain to the 
students using an acronym — IMSCI. “I” stands for immersion; by immersing students in text 
types or genres and analyzing the features of those genres, students gain the familiarity they need 
to imagine the goal of a writing task. “M” stands for modeling; the teacher actively models 
writing the text type or genre and models skills and strategies that are relevant to writing the 
genre. “S” stands for shared writing; teacher and students co-construct an example of the text 
type or genre, while incorporating relevant skills and strategies. “C” stands for collaborative; 
before asking students to try out the text type on their own, an intermediate step is to allow 
students to try writing the genre with a partner (this step is optional for some genres, such as 
autobiography). The final “I” stands for independent; once students have had the genre modeled 
and have participated in shared and/or collaborative writing of the genre, they are more likely to 
be ready to write in the genre independently. I emphasize, when teaching this model, that even 
when students are writing collaboratively or independently, they are never left entirely to their 
own devices. The teacher should be conferencing, offering guidance, answering questions, 
asking questions, and doing one-on-one or small group reteaching of relevant skills and 
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strategies. The IMSCI model is a way of conceptualizing gradual release of responsibility 
(Pearson and Gallagher, 1983) that is specific to writing instruction. 

The second focus of my instruction is experiential learning. The students live the 
experience of writing instruction based on the IMSCI model as I demonstrate how each element 
of the IMSCI can be used to teach a particular genre of writing and my students actively 
participate. When I model how to teach a particular prewriting strategy, they try out the strategy 
for themselves. In this way, my students experience the IMSCI model and reflect on that 
experience before they teach writing lessons in their practicum at local elementary schools where 
they teach lessons based on the IMSCI model. 

Most of the students’ final concept maps included the IMSCI model, though at varying 
levels of specificity (see Table 1). 


Table 1. Inclusion of IMSCI model in final concept maps. 


Included IMSCI 
and elements of 
IMSCI 

Included one or 
more elements of 
IMSCI 

Mentioned IMSCI, 
but provided no 
detail 

No mention of IMSCI, no 
mention of any elements of 
IMSCI 

Total 

48 

15 

5 

13 

81 

59% 

19% 

6% 

16% 



Although I taught the IMSCI model as declarative knowledge, by creating the lived experience 
for my students, my hope was that they would internalize the IMSCI model. I can see that 84% 
of the students did internalize the model to some degree, and that 59% of them internalized all 
elements of the IMSCI model. 

Another goal of my class was to ensure that students understood that writing is a non- 
linear, or recursive, process that involves pre-writing, drafting, revising, editing/proofreading, 
and publishing. Table 2 shows that 79% of my students included some aspect of the writing 
process in their final concept map. 


Table 2. Inclusion of writing process in final concept maps 


Included all elements 
of the writing process 

Mentions one or 
more elements of 
writing process 

Mentions writing as a 
process or “steps” 

No mention of writing 
process or elements of 
writing process 

Total 

53 

7 

4 

17 

81 

65% 

9% 

5% 

21% 



The goal is not, of course, to have students create detailed concepts maps, but rather for students 
to gain the pedagogical content knowledge they need to be successful teachers of writing. In the 
future, I plan to use concepts as a learning tool rather than as an assessment of their learning. I 
plan to have them create maps as a whole-class collaboration, in pairs, and individually. By 
comparing their maps with those of peers and with mine, students can evaluate their 
understanding in a low-risk context. 

IV. Implications and Recommendations. 

Maps are a flexible tool for both learning and assessment. They can be completed in class 
or as a homework assignment. One can broaden or narrow the scope of the map as needed. For 
example, I could ask students, while studying the best practices for teaching grammar to improve 
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writing, to map their knowledge of grammar instruction at the beginning of the unit so that I can 
assess where students are in their understanding. I could ask them to create another map after 
they have experienced research-based practices like sentence combining and after reading a 
position statement on grammar instruction from the National Council of Teachers of English. 

Concept mapping can be used in any course or discipline as a way for students to 
graphically represent their understanding of content. Because concepts are linked with lines and 
words or short phrases, the cognitive labor is different from writing an essay on the same topic 
where students must concentrate on organization, sentence structure, and grammatical 
conventions of writing, as well as on the concepts. A concept map liberates students from the 
concerns that accompany formal essay writing, allowing them to focus their whole attention on 
concepts and the relationships between and among those concepts. 

While concept maps are used in a variety of disciplines as an assessment device, the 
configuration of the task varies. In some studies (e.g. Ruiz-Primo, et al., 2001), the researcher 
provides students with some or all of the concepts, and the students’ task is to connect them 
logically. To examine students’ understanding of effective writing instruction, it might be useful 
to compare a free recall concept map with solicited recall for which certain elements of concept 
are provided to the students. 

It could also be useful to compare the results of concept maps with written essays, both 
done in response to the same prompt. Research has shown that concept maps aid students’ essay 
writing (Parkes, et al., 2000), but how students use those maps to support their essay writing is 
unclear. Also, if a student’s concept map is complex, will that complexity be reflected in his or 
her essay? Conversely, can a student write an essay that captures the complexity of the concepts 
if his or her concept map is not complex? 

V. Conclusions. 

The potential of concept mapping as a way to measure student learning has been 
demonstrated in many disciplines. I was able to validate, using concept maps, that students 
developed more complex constructions of their knowledge (Kegan, 2000) about effective writing 
instruction. Students’ post-course concept maps showed that pedagogical content knowledge 
(Shulman, 1987) had become integrated into their understanding of effective writing instruction. 
Practitioner research of this kind allows us to treat teaching as “community property” (Shulman, 
2004), opening the doors of our classrooms so that our teaching methods and forms of 
assessment can be shared with and critiqued by others. 

References 

Artiles, A. J. and McClafferty, K. (1998). Learning to teach culturally diverse learners: Charting 
change in preservice teachers' thinking about effective teaching. The Elementary School Journal, 
95(3), 189-220. 

Bass, R. (1999). The scholarship of teaching: What’s the problem? Inventio: Creative Thinking 
about Learning and Teaching, 7(1), 1-9. 

Beyerbach, B. A. (1988). Developing a technical vocabulary on teacher planning: Preservice 
teachers' concept maps. Teaching and Teacher Education, 4, 339-347. 


Journal of the Scholarship of Teaching and Learning, Vol. 8, No. 3, October 2008. 


127 



Read, S. 


Bolte, L. A. (1999). Using concept maps and interpretive essays for assessment in mathematics. 
School Science and Mathematics, 99(1), 19-30. 

Borko, H., Liston, D., Whitcomb, J. A. (2007). Genres of empirical research in teacher 
education. Journal of Teacher Education, 55(1), 3-11. 

Graves, D. H. (2002). Testing is not teaching: What should count in education. Portsmouth, NH: 
Heinemann. 

Jacobs-Lawson, J. M. and Hershey, D. A. (2002). Concept maps as an assessment tool in 
psychology courses. Teaching of Psychology, 29(1), 25-29. 

Jones, M. G. and Vesilind, E. M. (1996). Putting practice into theory: Changes in the 
organization of preservice teachers pedagogical knowledge. American Educational Research 
Journal, 33(1), 91-117. 

Kegan, R. (2000). What ‘form’ transforms: A constructive-developmental perspective on 
transformational learning. In J. Mezirow and Assoc, (eds.). Learning as transformation: Critical 
perspectives on a theory in progress. San Francisco: Jossey-Bass. 

Kirby, S. N. McCombs, J. S., Barney, H., and Naftel, S. (2006) Reforming teacher education : 
something old, something new. Santa Monica, CA: Rand Corporation. 

Lavigne, N. C. (2005). Mutually informative measures of knowledge: Concept maps plus 
problem sorts in statistics. Educational Assessment, 10(1), 39-71. 

Markham, K., Mintzes, J., and Jones, M. G. (1994). The concept map as a research and 
evaluation tool: Further evidence of validity. Journal of Research in Science Teaching, 37(1), 
91-101. 

Mintzes, J. J., Wandersee, J. H., and Novak, J. D. (2001). Assessing understanding in biology. 
Journal of Biological Education, 35(3), 118-124. 

Moats, L. C. and Foorman, B. R. (2003). Measuring teachers’ content knowledge of language 
and reading. Annals of Dyslexia, 53, 23-45. 

Murphy, P. (2003). Discovering the end in the beginning. Language Arts, 80(6), 461-469. 

Napoli, M. (2001). Presendee teachers ’ reflections about learning to teach writing. Unpublished 
manuscript, Pennsylvania State University, University Park. (ERIC Document Reproduction 
Service No. ED459472) 

Novak, J.D. (1998). Learning, creating, and using knowledge: Concept maps as facilitative tools 
in schools and corporations. Mahwah, NJ: Erlbaum. 


Journal of the Scholarship of Teaching and Learning, Vol. 8, No. 3, October 2008. 


128 



Read, S. 


Novak, J. and Gowin, B. 1984. Learning how to learn. Cambridge: Cambridge University Press. 

Parkes, J., Zimmaro, D. M., Zappe, S. M., and Suen, H. K. (2000). Reducing tak-related 
varieance in performance assessment using concept maps. Educational Research and Evaluation, 
6(4), 357-378. 

Pearson, P.D., and Gallagher, M.C. (1983). The instruction of reading comprehension. 
Contemporary Educational Psychology, 8, 317-344. 

Phelps, G. and Schilling, S. (2004). Developing measures of content knowledge for teaching 
reading. The Elementary School Journal, 105, 3 1-48 

Reutzel, D. R., Dole, J. A., Sudweeks, R., Fawson, P., Read, S., Smith, J., Donaldson, R., Jones, 
C., and Herman, K. (2007). “Developing the Literacy Instruction Knowledge Scales (LIKS): A 
Comprehensive Assessment of Primary Grade Teachers’ Knowledge of Reading and Writing 
Instruction,” Paper presented at the Annual Conference of the American Educational Research 
Association, April, 2007. Chicago, IL. 

Shavelson, R. and Ruiz-Primo, M. (2000). On the psychometrics of assessing science 
understanding. In Mintzes, J. J., Wandersee, J. H., and Novak, J. D. (Eds.) Assessing science 
understanding: A human constructivist view. San Diego, CA: Academic Press. 

Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard 
Educational Review, 57, 1-22. 

Shulman, L. S. (2004). Teaching as community property: Essays on higher education. San 
Francisco: Jossey-Bass. 

Sims-Knight, J. E., Upchurch, R. L., Pendergrass, N., Meressi, T., Fortier, P., Tchimev, P., 
VonderHeide, R. and Page, M. (2004). Using concepts maps to assess design process knowledge. 
Frontiers in Education, 2004. FIE 2004. 34th Annual, 2, 20-23. 

Walker, J. M. T., King, P. H., (2003). Concept Mapping as a Form of Student Assessment and 
Instruction in the Domain of Biomedical Engineering, Journal of Engineering Education, 19(2), 
167-179. 

West, D. C., Park, J. K., Pomeroy, J. R., and Sandoval, J. (2002). Concept mapping assessment 
in medical education: A comparison of two scoring systems. Medical Education, 36, 820-826. 

Williams, M. (2004). Concept mapping-a strategy for assessment. Nursing Standard, 19(9), 33- 
38. 


Journal of the Scholarship of Teaching and Learning, Vol. 8, No. 3, October 2008. 


129 



